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Abstract Body 
Background/context: 

Description of prior research and/or its intellectual context and/or its policy context. 

Data from the National Assessment of Educational Progress (NAEP) show that in 8th grade, at a 
time when all students are expected to be able to acquire information through the reading of 
textbooks, 34 percent of boys and 24 percent of girls cannot read at the basic level (Grigg, 
Donahue, &, Dion, 2007; Lee, Grigg, &, Donahue, 2007). Among Black, and Hispanic students, 
the picture is even more discouraging: 49 percent of Black, and 45 percent of Hispanic 8th 
graders read below the basic level. Thus, by the time they reach high school, certain students 
have already been "left behind." NAEP data also indicate that 27 percent of 12th-grade public 
school students scored below the basic level and only 35 percent scored above the proficient 
level. Since 1992, NAEP scores have declined significantly in all reading contexts (i.e., reading 
for literary experience, reading for information, reading to perform a task). These findings 
reflect an urgent need to identify effective scientifically based programs for improving reading in 
high schools in the US. 

Purpose/objective/research question/focus of study: 

Description of what the research focused on and why. 

Rigorous research provides information that will allow other schools and districts to select 
interventions that have a scientifically based track record of effectiveness. All Striving Reader 
grants include the mandate to evaluate literacy intervention(s) targeted to adolescents who are 
reading significantly below grade level. Although all studies followed the same guidelines for 
evaluation, each was implemented in a different context, evaluated different interventions or 
curricula, and utilized different evaluation designs. This symposium will bring together five 
evaluations to share information about the challenges in both designing and implementing the 
designs of such studies after two years of implementation. In addition, the consideration of 
implementation context in the interpretation of the results and what should be done when 
planning to implement will also be discussed. As a result evaluators hope to provide important 
information to those planning similar studies in school districts and other complex social 
settings. 

Setting: 

Specific description of where the research took place. 

Springfield/Chicopee: In the 2006-07 school year, all five schools were eligible for Title I. In 
Springfield, twenty-nine percent of the students were African American, 52 percent were 
Hispanic, and 14 percent were white; 71 percent were identified as low-income, and 13 percent 
were identified as Limited English Proficiency (LEP) students. In Chicopee, 78 percent were 
white, 3 percent were African American, and 18 percent were Hispanic; 35 percent were 
identified as low-income, and less than 2 percent were identified as LEP students. 

Newark: Nineteen middle schools participate and were selected because they 1) were eligible for 
Title I funding, 2) served a minimum of two grades across sixth-, seventh- and eighth-grades, 3) 
were not already using READ 180, 4) were categorized as “in need of improvement” under No 
Child Left Behind, and 5) served a minimum of 25 eligible students reading at least two grades 
below grade level, based on the 2006 New Jersey state assessment. In these schools, 58 percent 
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of students are African American, 41 percent are Hispanic, 88 percent are identified as low- 
income, and 7 percent are identified as being English Language Learners (ELL). 

Memphis: Eight middle schools serving over 6000 students in grades six through eight in 
Memphis, Tennessee are participating in the Striving Readers project. Ninety-five percent of the 
students served by these schools are African American and 5 percent are Hispanic. Eighty-eight 
percent of these students are eligible for free or reduced priced lunch, and 3 percent are identified 
as English Language Learners. 

Ohio: The Ohio Department of Youth Services (ODYS) system is made up of seven youth 
detention facilities, six designated male facilities and one a female facility, ah participating. 
Students are assigned to a facility based on gender, type of offense and availability of space. 

High schools are located within each facility; a small number of students are middle school age. 
In May 2006, 1,628 students were enrolled. Seven percent of those students were female, 48 
percent were African American, and 46 percent were white. The average length of stay for a 
student is about 10.5 months, although this varies substantially across students. 

Portland: Lour high schools, three middle schools, and three K-8 schools currently participate in 
the Portland Striving Readers grant. All 10 of these schools receive Title I funding; 65 percent of 
the students are identified as low income. On average, 28 percent of the students served by these 
schools are reading at least two years below grade level. None of the schools had achieved 
Adequate Yearly Progress at the time of the Striving Readers application in 2005. Lifteen percent 
of the students in the 10 schools are identified as English Language Learners, and 59 percent are 
non-white. 

Population/Participants/Subjects: 

Description of participants in the study: who (or what) how many, key features (or characteristics). 

All Striving Reader grants include students who are reading two levels below their grade level 
(in some cases students who are not below a fourth grade reading level). 

Springfield/Chicopee: Ninth-grade students are eligible and randomly assigned to one of the two 
targeted interventions or to the control condition: equally distributed among groups. Students are 
screened using the Scholastic Reading Inventory. Of the 334 students eligible in 2006-07, 285 
were placed in the targeted interventions or in the control group in the fall. 

Newark: The New Jersey State Language Arts assessment determines eligibility; special 
education students who meet district’s eligibility requirements are included. Nineteen schools 
were randomly assigned; ten to implement READ 1 80 and nine to the control condition. In the 
first year of the study, 1,371 students participated in the evaluation. 

Memphis: Eligible students score in the bottom quartile of the state assessment, the Tennessee 
Comprehensive Assessment Program (TCAP). In the first year of the study, 1740 struggling 
readers were randomized to the treatment or control group. 

Ohio: Ah students incarcerated in the seven ODYS facilities who score below reading level, as 
measured by the Scholastic Reading Inventory (SRI), are eligible. In the first year of the study, 
795 eligible students were randomly placed but only 461 receive instruction due intake 
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complications. The remaining students were either released early, were on lock down, or did not 
participate. 

Portland: Eligible students’ grades 7 through 10 were placed using the Oregon State Assessment 
Test or the Group Reading Assessment and Diagnostic Evaluation. In the first year of the study, 
1,31 1 students were randomly assigned and 586 students were included (had both pre- and post- 
test scores). 

Intervention/Program/Practice: 

Specific description of the intervention, including what it was, how it was administered, and its duration. 

The programs presented include versions of READ 180, developed by Scholastic Inc., and 
Xtreme Reading, developed by the University of Kansas Center for Research on Learning. In 
Springfield-Chicopee and Memphis, these programs were supplemental to the regular language 
arts curriculum; in the Newark, Ohio, and Portland they were incorporated into the regular arts 
curriculum as well as offered as supplements. 

The READ 180 program is used to address the individual needs of struggling adolescent readers 
who are reading below grade level through adaptive and instructional software, teacher-directed 
instructional rotations, and the use of tailored textbooks and independent or modeled reading of 
high interest literature. The program focuses on elements of phonics, fluency, vocabulary, 
comprehension, spelling, writing and grammar, and promoting self-directed learning. The 
Xtreme Reading program represents one of the levels of instruction in the Content Literacy 
Continuum (CLC), the framework within the Strategic Instruction Model (SIM). The Xtreme 
Reading program (level three of the framework) focuses on strategy instruction, particularly 
reading instruction that helps struggling middle and high school students to develop accurate 
word recognition and increased fluency and comprehension. Xtreme Reading’s core 
instructional approaches include direct instruction, teacher modeling, paired student practice, and 
independent practice. 

Springfield/Chicopee: Teachers receive approximately 42 hours of training and coaching visits, 
generally one per month, for each intervention. READ 1 80 teachers were to receive a two-day 
initial training, a one-day follow-up training and an additional 8 seminars. Xtreme teachers were 
to receive a three-day initial training and four days of follow-up training during the year are 
provided. 

Newark: The Newark Striving Readers project modified READ 1 80 to include some 
supplemental instruction aligned with the state assessment. For eligible students, the READ 180 
curriculum replaces the district’s regular core language arts curriculum. Teachers receive 20 
hours of group training, five hours of training on using student assessment data to differentiate 
instruction, plus in-school coaching visits on an as-needed basis. 

Memphis: Teachers were to participate in two all-day summer training sessions, and up to seven 
two-hour training sessions during the year. Teachers also had access to online professional 
development material and ongoing classroom support as needed. Students can receive the 
targeted intervention for two years. 
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Ohio: New teachers were hired to teach the Read 180 program; they were expected to attend 20 
hours of professional development. In ODYS, Read 180 replaces the regular 45-minute literacy 
class for eligible students randomized to the treatment group. In addition, these treatment group 
students will also receive 45 -minutes of additional instruction using the model, for a total of a 
90-minute instructional block each day. 

Portland: Professional development is offered to both participating teachers and literacy coaches 
for 7 days during the summer and monthly half-days during the academic year. In the first 2 
years of the project, 2 teachers co-taught the Xtreme Reading class along with a content class 
such as language arts or social studies. Each school had 1 Xtreme Reading class per grade level, 
taught by the same pair of teachers (1 reading specialist and 1 content specialist). 

Research Design: 

Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic 
essay, randomized field trial). 

Each study is a randomized controlled field trial; Newark is a cluster-randomized field trial. The 
primary research question is whether or not the interventions or curricula implemented in these 
settings is effective in improving adolescent reading achievement for the targeted group as 
compared to the control group. 

Springfield/Chicopee: Eligible ninth grade students receive the standard ELA curriculum. 
Eligible students are randomly assigned to participate in one of the two supplemental programs 
(READ 180 or Xtreme Reading) or to “business-as-usual,” which may consist of tutoring or an 
elective in addition to the “business-as-usual” standard ELA curriculum. Eligible teachers are 
also randomly assigned to teach READ 1 80, Xtreme Reading, or the control group. 

Newark: Schools were randomly assigned to either implement READ 180 or to continue to 
provide only the regular language arts curriculum. Schools were blocked on four school-level 
variables prior to randomization: 1) school size, 2) proportion of ELL students, 3) proportion of 
students with special needs, and 4) AYP status. 

Memphis: Students scoring in the lowest quartile on the English/Language Arts component of 
the TCAP are randomly assigned to receive the supplemental Read 180 class or to a control 
group. Students continue to receive instruction in the regular language arts curriculum. Students 
in the treatment group receive READ 180 in addition to the regular language arts instruction. 

Ohio: Incoming students scoring below reading level, as measured by the Scholastic Reading 
Inventory (SRI), are randomly assigned to either the Read 1 80 class or to a control group. 
Students maintain their assignment until released from the facility. 

Portland: Prior to the start of each school year, eligible students in Grades 7-10 are randomly 
assigned to either the Xtreme Reading group or the control group. For Grades 7-8, the control 
group participates in the regular language arts curriculum; for Grades 9-10, the control group 
students participate in another elective course of their choosing. 
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Data Collection and Analysis: 

Description of plan for collecting and analyzing data, including description of data. 

Data are collected in all sites by the program which follows specified testing protocols. All sites 
have specified the use of multilevel modeling to account for the clustering of data with the 
exception of Springfield/Chicopee and Portland given those designs (they model the impacts of 
interventions on student outcomes using ANCOVA). All models will test intervention effects on 
achievement scores (students random assignment to group), controlling for pre -intervention 
reading levels and other covariates including school and cohort. A variety of covariates will be 
tested for inclusion in the analytical models. Where appropriate, analyses will also be run to 
disaggregate effects by grade level, gender, ELL, and special education. The outcome measures 
are as follows: 

Springfield/Chicopee: Stanford Diagnostic Reading Test 4 

Newark: Stanford Achievement Test-10, New Jersey State Language Arts assessment 
Memphis: Iowa Test of Basic Skills, Tennessee Comprehensive Assessment Program 
Ohio: California Assessment Test, Scholastic Reading Inventory 

Portland: Oregon State Assessment Test, Group Reading Assessment Diagnostic Evaluation 

Findings/Results: 

Description of main findings with specific details. 

Lirst year finding across projects were fairly consistent in that preliminary impact results were 
observed for students in middle schools but not high schools. However, many of the studies 
relied on more than one cohort for analytic power to detect effects. These papers will present the 
analysis of over two years of implementation data to be reported to the Department of Education 
in early 2009. In Springfield/Chicopee, both interventions depend on a minimum of three 
cohorts of grade 9 students to assess effectiveness with appropriate power. In Newark, the 
impact of READ 180 on eligible students after one year, and after two years of implementation. 
The results that we propose to present will be based on an ITT analysis fitting a linear two-level 
model with student and school as the two levels. At the first level of the model, achievement for 
students within schools is predicted by a series of student characteristics. In Ohio, students in 
Read 180 for three consecutive terms have gained approximately 75 Lexile points on the SRI 
measure. This rate of growth is significantly beyond the rate of growth historically achieved by 
these youths. 

Conclusions: 

Description of conclusions and recommendations of author(s) based on findings and over study. (To support the 
theme of 2009 conference, authors are asked to describe how their conclusions and recommendations might inform 
one or more of the above noted decisions — curriculum, teaching and teaching quality, school organization, and 
education policy .) 

Many schools are struggling to meet the needs of students who lack the reading skills required to 
succeed in high school and beyond. Because schools have finite resources, it is crucial that 
administrators allocate financial and human capital to literacy interventions promising the 
greatest potential for success. These studies are considered critical by the Department of 
Education to contribute to the field of adolescent literacy by addressing the primary research 
question of what interventions are effective. In addition, these studies are also some of the first 
to address what methods should be employed in such RCT trials including what is required in 
terms of policy for implementation and collaboration to conduct such efforts. Given the steady 
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push for more rigorous research and a corresponding dearth of information on this particular 
group of students, these papers add significantly to our understanding of both the benefits of an 
experimental research design and the latest research in the field of adolescent literacy. The 
presenters argue that while there are substantial challenges to conducting rigorous experiments in 
a school setting, the results can advance our understanding of both practical research 
methodology and the issue studied. The goal of this symposium is to present information 
regarding the benefits of such studies in the context of their complications. 
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